On knowledge transfer in object class recognition

In recent years, impressive results have been reported for the recognition of individual object classes, based on the combination of robust visual features with powerful statistical learning techniques. As a result, the simultaneous recognition of many object classes is coming into focus, posing challenges with respect to both model complexity and the need for increasing amounts of training data. Reusing once acquired information in the context of related recognition tasks, effectively transferring knowledge between object classes, has been identified as a promising route towards scalable recognition. Besides increasing scalability, knowledge transfer has been shown to enable novel tasks, such as the recognition of object classes for which no training data are available, termed zero-shot recognition. In this case, missing training data is compensated by exploiting additional, complementary sources of knowledge, such as linguistic knowledge bases. Based on these encouraging prospects, this thesis explores four different dimensions of knowledge transfer in object class recognition. First, we investigate the role of visual features as a low level representation of transferable knowledge. Based on an extensive evaluation of existing state-ofthe-art local feature detectors and descriptors, we identify shape-based features in connection with powerful spatial models as a promising candidate representation. Building upon this result, we further introduce a novel flavor of local shape-based features, as well as a generic appearance descriptor based on shading artifacts. Second, we highlight the connection between knowledge transfer and generalization across basic-level object categories, by recognizing objects according to potential functions or affordances. In particular, we demonstrate that visually distinct hints on affordances, modeled as collections of local shape features, can be shared and hence transfered between object classes. Third, we design shape-based object class models for knowledge transfer, representing object classes as spatially constrained assemblies of parts, including pair-wise symmetry relations. These models are both compositional and incremental, allowing for knowledge transfer either on the level of entire object class models or restricted to a subset of model components. While knowledge transfer in these models has to be guided by manual supervision, we demonstrate the benefit of knowledge transfer for object class recognition when learning from scarce training data. And fourth, we demonstrate that exploiting additional sources of knowledge besides real world training images can aid object class recognition, effectively transferring knowledge between different representations. In particular, we use linguistic knowledge bases in connection with semantic relatedness measures to automatically determine potential sources and targets of knowledge transfer for zero-shot recognition, and show the successful learning of shape-based object class models from collections of 3D computer aided design (CAD) models, not using any real world